[Security Solution][Endpoint] Endpoint generator and data loader support for Host Isolation #100813

paul-tavares · 2021-05-27T18:43:40Z

Summary

Re-introduces commit 57f59bd which was backed out due to a bug

Original Commit: #100727

new data generator for both Isolate and UnIsolate Fleet actions
Endpoint test/dev data loader was enhanced to load actions for each endpoint (aka: agent) when run with the --fleet option

…endpoint generator and test data loader (elastic#100727) * Generate random isolation values for endpoint metadata * Generator for Fleet Actions * Added creation of actions to the index test data loader (cherry picked from commit 57f59bd)

… numbers

elasticmachine · 2021-05-27T18:43:43Z

Pinging @elastic/security-onboarding-and-lifecycle-mgt (Team:Onboarding and Lifecycle Mgt)

paul-tavares · 2021-05-27T18:44:59Z

x-pack/plugins/security_solution/common/endpoint/data_generators/base_data_generator.ts

+
+  /** Generate either `true` or `false` */
+  protected randomBoolean(): boolean {
+    return this.random() < 0.5;


the bug fix for the problem that was introduced originally is here. Instead of using Math.random(), we need to use this.random() which is a seeded random number generator

…-generator-changes

paul-tavares · 2021-05-28T12:35:45Z

@oatkiller , @jonathan-buttner ,
Pinging both of you based on who last touched the resolver test file x-pack/plugins/security_solution/public/resolver/models/indexed_process_tree/isometric_taxi_layout.test.ts. I had to update the snapshot because the generation of the Endpoint Host metadata is making an additional call to the seeded random number generator (this.random() which used seedrandom package). Because of that additional call, some of the resolver tests failed because the order changed.

In talking with @pzl , having the data generator tied/coupled to tests seems troublesome. We might break out the generation of the Endpoint Host metadata into a separate generator (similar to the new one under x-pack/plugins/security_solution/common/endpoint/data_generators/**) so that we don't impact these test cases going forward.

jonathan-buttner · 2021-05-28T13:01:58Z

@oatkiller , @jonathan-buttner ,
Pinging both of you based on who last touched the resolver test file x-pack/plugins/security_solution/public/resolver/models/indexed_process_tree/isometric_taxi_layout.test.ts. I had to update the snapshot because the generation of the Endpoint Host metadata is making an additional call to the seeded random number generator (this.random() which used seedrandom package). Because of that additional call, some of the resolver tests failed because the order changed.

In talking with @pzl , having the data generator tied/coupled to tests seems troublesome. We might break out the generation of the Endpoint Host metadata into a separate generator (similar to the new one under x-pack/plugins/security_solution/common/endpoint/data_generators/**) so that we don't impact these test cases going forward.

Yeah that might have been a poor choice on my part 😬 . Maybe we should hard code the values in the tests. It was just really convenient to have the generator create the data that we needed at the time.

paul-tavares · 2021-05-28T14:12:13Z

@jonathan-buttner thanks for the review. No worries. Yeah, we have used the endpoint generator in our tests, but usually will statically set the properties we're looking to test. we also try not to use snapshots too much because of issues like this 😄

Thanks again

kibanamachine · 2021-05-28T14:32:57Z

💛 Build succeeded, but was flaky

continuous-integration/kibana-ci/pull-request
Commit: 83b8f03
Storybooks Preview
Flaky suites:
- xpack-securitySolutionCypressChrome

Test Failures

Kibana Pipeline / general / "before all" hook for "should not allow user with read only privileges to attach alerts to cases".Alerts timeline Privileges: read only "before all" hook for "should not allow user with read only privileges to attach alerts to cases"

Link to Jenkins

Stack Trace

Failed Tests Reporter:
  - Test has not failed recently on tracked branches

AssertionError: Timed out retrying after 60000ms: Expected <span.euiLoadingContent> not to exist in the DOM, but it was continuously found.

Because this error occurred during a `before all` hook we are skipping the remaining tests in the current suite: `Alerts timeline`

Although you have test retries enabled, we do not retry tests when `before all` or `after all` hooks fail
    at Object.waitForAlertsPanelToBeLoaded (http://localhost:61161/__cypress/tests?p=cypress/integration/detection_alerts/attach_to_case.spec.ts:19450:43)
    at Context.eval (http://localhost:61161/__cypress/tests?p=cypress/integration/detection_alerts/attach_to_case.spec.ts:18539:18)

Metrics [docs]

Unknown metric groups

References to deprecated APIs

id	before	after	diff
`fleet`	22	20	-2

History

💔 Build #127747 failed a34cc45

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @paul-tavares

…ort for Host Isolation (elastic#100813) Re-introduces the changes from elastic#100727 which was backed out due to a bug. Changes included: * Generate random isolation values for endpoint metadata * Generator for Fleet Actions * Added creation of actions to the index test data loader Plus: * Fix generator `randomBoolean()` to ensure it works with seeded random numbers * Update resolver snapshots due to additional call to randomizer

kibanamachine · 2021-05-28T14:43:47Z

💚 Backport successful

Status	Branch	Result
✅	7.x

This backport PR will be merged automatically after passing CI.

…ort for Host Isolation (#100813) (#100904) Re-introduces the changes from #100727 which was backed out due to a bug. Changes included: * Generate random isolation values for endpoint metadata * Generator for Fleet Actions * Added creation of actions to the index test data loader Plus: * Fix generator `randomBoolean()` to ensure it works with seeded random numbers * Update resolver snapshots due to additional call to randomizer Co-authored-by: Paul Tavares <[email protected]>

* master: (77 commits) [RAC][Security Solution] Register Security Detection Rules with Rule Registry (elastic#96015) [Enterprise Search] Log warning for Kibana/EntSearch version mismatches (elastic#100809) updating the saved objects test to include more saved object types (elastic#100828) [ML] Fix categorization job view examples link when datafeed uses multiple indices (elastic#100789) Fixing ES archive mapping failure (elastic#100835) Fix bug with Observability > APM header navigation (elastic#100845) [Security Solution][Endpoint] Add event filters summary card to the fleet endpoint tab (elastic#100668) [Actions] Taking space id into account when creating email footer link (elastic#100734) Ensure comments on parameters in arrow functions are captured in the docs and ci metrics. (elastic#100823) [Security Solution] Improve find rule and find rule status route performance (elastic#99678) [DOCS] Adds video to introduction (elastic#100906) [Fleet] Improve combo box for fleet settings (elastic#100603) [Security Solution][Endpoint] Endpoint generator and data loader support for Host Isolation (elastic#100813) [DOCS] Adds Lens video (elastic#100898) [TSVB] [Table tab] Fix "Math" aggregation (elastic#100765) chore(NA): moving @kbn/io-ts-utils into bazel (elastic#100810) [Alerting] Adding feature flag for enabling/disabling rule import and export (elastic#100718) [TSVB] Fix Upgrading from 7.12.1 to 7.13.0 breaks TSVB (elastic#100864) [Lens] Adds dynamic table cell coloring (elastic#95217) [Security Solution][Endpoint] Do not display searchbar in security-trusted apps if there are no items (elastic#100853) ...

paul-tavares and others added 2 commits May 27, 2021 14:11

Fix generator randomBoolean() to ensure it works with seeded random…

a34cc45

… numbers

paul-tavares added v8.0.0 release_note:skip Skip the PR/issue when compiling release notes Team:Defend Workflows “EDR Workflows” sub-team of Security Solution v7.14.0 auto-backport Deprecated - use backport:version if exact versions are needed labels May 27, 2021

paul-tavares self-assigned this May 27, 2021

paul-tavares requested a review from a team as a code owner May 27, 2021 18:43

paul-tavares commented May 27, 2021

View reviewed changes

kevinlog approved these changes May 27, 2021

View reviewed changes

paul-tavares added 2 commits May 28, 2021 08:27

Update resolver snapshots due to additional call to randomizer

58fa724

Merge remote-tracking branch 'upstream/master' into fix/kibana-100800…

83b8f03

…-generator-changes

jonathan-buttner approved these changes May 28, 2021

View reviewed changes

paul-tavares merged commit e3517ed into elastic:master May 28, 2021

paul-tavares deleted the fix/kibana-100800-generator-changes branch May 28, 2021 14:40

kibanamachine mentioned this pull request May 28, 2021

[7.x] [Security Solution][Endpoint] Endpoint generator and data loader support for Host Isolation (#100813) #100904

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Security Solution][Endpoint] Endpoint generator and data loader support for Host Isolation #100813

[Security Solution][Endpoint] Endpoint generator and data loader support for Host Isolation #100813

paul-tavares commented May 27, 2021

elasticmachine commented May 27, 2021

paul-tavares May 27, 2021

paul-tavares commented May 28, 2021

jonathan-buttner commented May 28, 2021

paul-tavares commented May 28, 2021

kibanamachine commented May 28, 2021

Stack Trace

References to deprecated APIs

kibanamachine commented May 28, 2021

[Security Solution][Endpoint] Endpoint generator and data loader support for Host Isolation #100813

[Security Solution][Endpoint] Endpoint generator and data loader support for Host Isolation #100813

Conversation

paul-tavares commented May 27, 2021

Summary

elasticmachine commented May 27, 2021

paul-tavares May 27, 2021

Choose a reason for hiding this comment

paul-tavares commented May 28, 2021

jonathan-buttner commented May 28, 2021

paul-tavares commented May 28, 2021

kibanamachine commented May 28, 2021

💛 Build succeeded, but was flaky

Test Failures

Stack Trace

Metrics [docs]

References to deprecated APIs

History

kibanamachine commented May 28, 2021

💚 Backport successful